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Abstract: The micro facial expressions and eye blinks of the liar are analyzed by the lie detection 
system, which makes use of the Facial Landmark Detection System that is included in the OpenCV Tool 
Kit. While the suspect responds to a series of questions, the system will observe the motions of the 
facial muscles and the rate at which their eyes blink. The Eye-Opening Ratio is used to determine the 
eye-opening in each frame. An approach that makes use of human behaviors to identify deception has 
been proposed here. In order to assess whether a candidate is being dishonest during an interrogation 
and come up with a conclusion about them, the system will do face detection and an eye blink 
calculation. During an interrogation session, the interrogator can use this result to assist them in doing 
an analysis of the blink threshold value and locating the lie. In the future, developments could include 
thermal monitoring, which would involve collecting video of the suspect while they are answering 
questions during interrogation. This video would then be used in conjunction with face detection and 
eye blink rate to provide a more in-depth analysis of the suspect's dishonest behavior. 


Keywords: Facial Micro-Expressions, Eye Blinks, Tool Kit, Facial Landmark, OpenCV, EAR, Machine 
Learning. 


Introduction 


Interrogation often includes some form of lie detection and investigation. Lie detection is typically carried out 
through the use of a technique known as deceptive facial expression recognition, on the basis of which the 
degree of deceit is determined [1]. In addition to this way of detecting nervousness through a person's facial 
expression, the interrogator also analyses the subject's body language, eye contact, and other habits to 
determine the extent to which they are apprehensive [2-5]. Despite the fact that it is an effective strategy for 
detecting lies throughout the questioning process, there are situations when it is ineffective [6]. This is the 
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primary justification for why a more sophisticated procedure ought to be used in order to ensure accuracy in 
the Deceptive reading process. It is essential for the success of such an interrogation process to carry out an 
accurate false reading while the suspect is being questioned [7-13]. 


Interaction between humans has played the most important part in the evolution of communication in its 
traditional forms. There is a voice recognition capability built into artificial intelligence. Using this 
technology, artificial intelligence is able to recognize the voice of a human, communicate with schedules, and 
perform other tasks such as playing music [14]. These interactions with humans have the potential to reduce 
the amount of labour that is performed by the user. Today, the human face includes more significant 
information that is used in day-to-day life, such as utilising face recognition to unlock a phone [15-18]. This 
information is used on a daily basis. There is more data associated with face recognition, such as the user's 
current emotional state. Recently, machine learning has been utilised in facial recognition, which has enabled 
it to progress to the next level [19]. The facial expression of the user can be located by the recognition system 
with remarkable precision. Different facial expressions, such as happiness, fear, sadness, anger, and so on, 
may all be discovered in machine learning. The Haar Cascade algorithm was utilised in order to discover 
facial expressions [20]. This algorithm is comprised of a set of positive images and a set of negative images, 
both of which are compared with the real-time face of a human being in order to determine the expression that 
the human being is currently displaying [21]. Python's "dlib" library is utilised in the process of recognizing 
human faces and facial landmarks in order to do so. The 68 facial coordinates represented here are the most 
important aspects of the subject's face [22-25]. All of the facial landmarks’ eyes, eyebrows, nose, and mouth 
are depicted as curves, and the recognition of the face is done in the box that is located outside the inner one 
[26-33]. 


It is fundamentally necessary to recognise the face and interpret the misleading eye blink. During an 
interrogation, this technology is able to calculate the eye blink rate of the deceptive individual by applying a 
formula called EAR [34-41]. In order to interpret the candidates’ levels of anxiousness and anxiety, eye- 
tracking technology and estimates of eye-blinking rate are both used on the candidates [42]. This gives the 
interrogator a critical approach for basically reading the deceit in an accurate manner, which is a significant 
advantage. When questioning the dishonest party multiple times with a variety of inquiries, one can eventually 
arrive at a conclusive answer. The research effort that is presented here is intended to demonstrate the 
uniqueness and applicability of our research problem [43]. In the form of a written report, the goal is to 
acquire a grasp of the existing research as well as the controversies that are pertinent to this topic [44]. 


The electroencephalogram (EEG) is a method that does not include any intrusive procedures in order to 
measure the electrical activity of the brain along the scalp. An electroencephalogram is a recording that shows 
the continuing changes in these signals, which are measured using scalp electrodes as continuous variations in 
voltage over time [45-53]. This recording is also known as an EEG (EEG). A program known as Azure 
Machine Learning comes equipped with 25 different machine learning algorithms. It has the ability to process 
data for the rapid creation of machine learning algorithms and has a workflow that uses drag and drop. Azure 
offers a linear model that includes two-class classification and regression for predicting categories and values. 
This model is linear since it predicts categories using linear models [54]. 


The interrogation videos are analysed by artificial intelligence, which finds instances of lying [55]. It does this 
by analysing human micro-expressions like face movement, eyebrow movement, lip placement, and audio 
frequency in order to uncover voice patterns that show whether or not a person is lying [56-63]. It does this by 
employing human micro-expression detectors as well as audio frequency in a multi-model detector to identify 
dishonesty [64]. 
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Python's dlib package can identify recognisable features of the face [65]. The facial landmark is equipped with 
a 68-point detector that can identify places on the brow, nose, mouth, jawline, and eye of a person. The facial 
landmark can be detected by using OpenCV and Python in conjunction with the dlib module [66]. 


The technique of locating points within the face module is referred to as face landmark detection. OpenCV 
underwent the development and implementation of several algorithms [67]. The x-axis, y-axis, and z-axis 
directions are projected from the tip of the nose to determine the pose that is associated with the face module. 
This allows the researchers to visualize the direction in which the subject's face is pointing, and they present a 
method that uses an automated algorithm to differentiate between lies and truths [68-71]. Lies are separated 
from the truth with the help of online polygraph devices, which can also identify dishonesty in written form. 
The program is able to identify dishonest individuals based on their text [72-76]. 


The eye blink pattern is used to determine whether or not someone is telling the truth. Calculations are done in 
a normal condition to determine the eye blink pattern, which is then compared to the eye blink pattern in an 
interrogating state [77-82]. Because of this distinction, deception can be found. MATLAB uses the HAAR 
cascade method in order to accomplish eye blink detection. The HAAR Cascade algorithm incorporates both 
the positive and negative facial emotions that humans are capable of [83]. 


The technology of thermal imaging can be used to identify dishonesty by determining the degrees of stress that 
persons are experiencing as well as the heat that is generated from the face [84]. When a human is being 
dishonest, their forehead will experience an increase in warmth [85-91]. Detection of deceit is achieved 
through the application of machine learning techniques to the facial thermal imaging data set [92]. The k- 
nearest neighbor method, logistic regression, and decision tree are the three algorithms that are utilised in this 
process [93-101]. 


Since the beginning of time, deception has been an inherent component of everyday existence. Because of this, 
it has been a topic of hobby discussion in a wide variety of fields, including psychology. The purpose of this 
article is to present a contemporary review of the relevant literature and to reflect on the development of 
approaches for lie detection up to the present time [102]. The first component investigates ancient practices that 
were documented around one thousand years BC. Phrenology, graphology, and polygraph analysis are some of 
the technical methods described in the second section, which are entirely based on scientific principles [103- 
111]. After this, a series of more modern technologies, such as functional magnetic resonance imaging (FMRI), 
brain fingerprinting, and facial action coding systems (FACS), will be carried out in order to determine the next 
step [112-118]. After becoming familiar with the historical development of techniques for lie detection, we will 
now discuss the potential for brand new initiatives, not only in the field of designing new techniques but also 
for the research into lie detection itself, including its causes and the regulatory difficulties that are related to 
deception [119]. 


Lie detection is one of the most important topics discussed in "psychology and law," which, in turn, is one of 
the most important subfields in applied psychology [120-125]. In point of fact, it is not difficult to comprehend 
why it is essential to determine whether or not a person is being dishonest or telling the truth during police 
investigations, courtroom trials, interviews regarding border manipulation, intelligence interviews, and so on. 
Several different lie detection tools have been created by psychologists and other mental health professionals 
for the purpose of detecting valuable resource lies. This kind of technology covers the whole tractable 
spectrum, beginning with behavior, progressing through speech analysis and the measurement of peripheral 
physiological reactions, and culminating in the recording of brain activity [126-131]. The reader will become 
familiar with the primary lie-detecting equipment that has been employed to this day, thanks to this text. It is 
also possible to pay attention to the attempts that liars make to fool people who attempt to locate their 
falsehoods, as well as to the important technical difficulty of determining how to check the correctness of these 
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lie detection devices. In controlled laboratory settings, the accuracy of lie detection devices may be tested 
without any complications; nevertheless, lying in such situations is completely manufactured. Examinees, for 
instance, have been known to tell falsehoods for the purpose of the examination, and their dishonesty has been 
tolerated [132]. 


The problem with assessing the accuracy of lie detection equipment in real-life scenarios is that it is genuinely 
hard to discern with certainty when someone is lying and when they are telling the truth. This makes it difficult 
to determine whether or not the equipment is accurate. This page also addresses the many lie-detecting tools 
that are currently available [133-141]. The same problem affects all of the tools: a cue that was once 
exclusively connected with deception, like Pinocchio's growing nose, does no present any longer. This 
indicates that there isn't always a single unmarried cue (or a cluster of unmarried cues) that investigators may 
merely rely on. It also indicates that mistakes are frequently made when certain tools are utilized in a given 
situation. It is possible that lie detection without the use of instruments is even more difficult; however, there is 
a discussion going on about whether or not certain people have excellent capabilities to hit upon lies. Following 
a discussion of the "wizards in lie detection" controversy, the subject comes to a close with a look at some of 
the most recent and significant advances in the field of lie detection research. The primary obstacle is the 
difficulty that interviewees face when being questioned by investigators of dishonesty, which, in turn, makes 
research on lie detection easier to conduct, particularly in intelligence-related contexts. It poses specific 
concerns that have not been answered previously and to which conventional "police investigations" deception 
studies are unable to provide solutions because of their methodology [142-161]. 


The long-promised deception is uncovered by fictitious and actual machines, such as polygraphs and mental 
microscopes [162]. Now, using techniques such as functional magnetic resonance imaging (FMRI) and 
electroencephalography (EEG), neuroscientists appear to have found what American scientists, attorneys, and 
police officers have been looking almost a century: a lie detector that is flawless [163]. These technologies 
include polygraphs, lie detectors, and polygraph machines. Lie detection utilising neuroscience is not 
necessarily more accurate, as Melissa M. Littlefield demonstrates; rather, it is merely ideologies for 
approximately discovering the dishonesty in a certain situation [164]. 


Electroencephalography, sometimes known as EEG, is the most extreme kind of convection. The electrical 
activity of the mind can be read via EEG readings. Many researchers are moving to new locations all around 
the world in order to capitalize on this generation. The study of EEG provides useful information about the 
ways in which the mind signals paintings during a variety of activities and states of thinking. Lie detection is a 
rapidly developing field that is increasingly being utilised in the fight against crime. Conventionally, this is 
accomplished through the use of language analysis, the identification of face and facial and frame movement, 
training observation, and the study of voice strain [165-169]. The advancement of cognitive science and 
neuroscience has led to improvements in EEG analysis, which provides more accurate information on how the 
brain functions. Analyses of EEG indicators are made possible thanks to a device that is capable of learning 
classification methods such as SVM, K-Means, ANN, and Linear Classifiers [170]. The noise of the indicators 
that are obtained from EEG can be reduced through the use of methods such as the Fast Fourier Transform 
(FFT). The purpose of this investigation is to carry out a literature review on the aforementioned three linked 
domains [171-174]. "Lie detection made possible with the use of EEG and other technological devices" 
Measuring EEG warnings, EEG signal processing, Feature extraction, and the manner in which to employ EEG 
in the way to identify falsehoods are all covered in this evaluation study. An algorithm for categorising things, 
Methods, and approaches utilised in the investigation of EEG signs in order to carry out lie detection, an 
overview, and a conclusion is presented here. 
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The activity of turning the information that was acquired during analysis into a requirement document is 
referred to as the requirement specification. It consists of the tasks that must be completed in order to define 
the requirements and conditions that must be met in order to obtain the product, taking into account the 
requirements of all conceivable users. These prerequisites are broken down into two distinct categories, each 
of which is discussed in the next section. User requirements are abstract assertions of the system requirements 
written for the interrogator and end user of the system, who typically do not have comprehensive technical 
expertise of the system. The instructions should be displayed on the device before the interrogator arrives at 
the scene of the interrogation. The questioner ought to determine the least number of times in a certain period 
of time that the liar should blink their eyes. After the interrogation has been completed, a warning should 
appear that specifies the maximum number of times the dishonest person can blink their eyes. 


The system requirements are the more extensive specification of the user's needs, and they can sometimes 
serve as a contract between the user and the developer. They consist of a collection of system services and 
constraints that are specified in greater detail. It is composed of a list of the different pieces of hardware and 
software that are required to complete a task. 


The FER method, which stands for "Facial Expression Recognition," is what the system employs to locate the 
face. The Image Classification issue, which is at the heart of machine learning, requires that images be 
allocated algorithmically. Using the Autoencoder approach, you may build a face image of a person with a 
neutral expression by following these steps. The current system is able to extract face features from neutral 
and emotional photographs using this method, even though it lacks more modern algorithms for the CK+ 
(Extended Cohn-Kanade Database) and the Japanese Female Facial Expression Database. 


The polygraph is hampered by the fact that it is a cumbersome piece of machinery, which necessitates a 
greater investment of time and energy when operating it. It monitors and records a number of physiological 
parameters, including blood pressure, pulse rate, respiration rate, and skin conductivity, among others. 


The use of polygraph testing has been the subject of debate for a long time due to the fact that it takes a very 
long time to obtain findings, and there is no evidence to suggest that a particular pattern of physiological 
reactions is exclusive to lying. Even the most trustworthy individuals can have moments of nervousness when 
answering questions, which can lead to inaccurate results. 


The solution that is being presented makes use of a technology that will immediately begin searching for the 
face of the human volunteer as soon as they sit down in front of the camera. The program looks for a group of 
pixels that are grouped together in a manner that is similar to a human face. The next step is an examination of 
the geometric characteristics of the mouth, nose, and eyes. This information is utilised for the purpose of 
recognising various sorts of facial expressions. When questions are asked, the eye blinking of the person who 
is being dishonest will be measured using a technique called EAR (Eye Aspect Ratio). The conclusion that 
there is any hint of dishonesty is reached by this algorithm after it identifies any change in the volunteer's 
facial expression as well as any changes in the eye blinking pattern when the volunteer is answering 
questions. Following the completion of the questionnaire by the volunteer, the face landmark data set is 
incorporated into the development and instruction of our model. 


The process of establishing the architecture, components, modules, interfaces, and data for a system in such a 
way that it satisfies the criteria that have been outlined is known as design. The design of the system provides 
information about its architecture, as well as the functions and modules that are a part of it. The next sections 
provide an in-depth look at the architecture of our suggested model. 


The term "computer vision" refers to the category of applications that encompasses the process of face 
detection. It is the process by which algorithms are built and taught to correctly locate faces or objects (in 


© 2023, IJHCS | Research Parks Publishing (IDEAS Lab) www.researchparks.org | Page 22 


Copyright (c) 2023 Author (s). This is an open-access article distributed under the terms of Creative Commons 
Attribution License (CC BY).To view a copy of this license, visit https: //creativecommons.org/licenses/by/4.0/ 


\(C RESEARCH INTERNATIONAL JOURNAL OF HUMAN COMPUTING STUDIES 


PARKS https://journals.researchparks.org/index.php/IJHCS e-ISSN: 2615-8159 | p-ISSN: 2615-1898 
Volume: 05 Issue: 02 | Feb 2023 


object detection, which is a related system) in photographs. Face detection is a subset of object detection. These 
can either be taken from still images or in real-time from a video camera. One use of this technology is found 
in the safety and surveillance systems of airports. In order for the camera software to be able to recognise a 
face, it must first detect the face and determine its features before proceeding to make an identification. When 
Facebook provides tagging suggestions to identify people in photographs, it must first locate the face of the 
person it is trying to identify. Face detection is a requirement for the augmented reality feature on social media 
apps like Snapchat. This feature enables users to realistically wear dog face masks by combining them with 
other fancy effects. Another application for facial recognition is the security of face ID on smartphones. I 
developed and built a system in this project that is capable of locating faces in real-time. Classifiers are used in 
face detection; these are computer programs that can recognise either a face (1) or anything that is not a 
face(O). In order to achieve a higher level of precision, classifiers have been taught to recognise faces by 
analyzing anywhere from thousands to millions of photos. LBP (Local Binary Pattern) and Haar Cascades are 
the two classes of classifiers that are utilised by OpenCV. The latter classification system will be utilised by us. 


The pixels in the image are broken down into squares according to their function using a method called Haar 
Wavelet Analysis. In order to get a high degree of accuracy from what is referred to as "training data," this 
makes use of machine learning techniques. In order to compute the "features" that have been identified, 
"integral image" notions are used here. In order to generate effective classifiers, Haar Cascades make use of the 
Adaboost learning algorithm. This approach chooses, from a wide pool of features, a chosen few that are 
particularly essential. 


This is a condensed example of feature extraction, as well as the distinction between face detection and face 
recognition. Face detection is concerned with localising a person's face, whereas face recognition is concerned 
with determining who they are. 


As was noted previously, Haar Cascades is an application of machine learning techniques. These techniques 
involve the training of a function using a large number of examples of both positive and negative pictures. The 
procedure known as feature extraction occurs here in the algorithm. 


During the process of feature extraction, the algorithm consults its stored training data in order to determine 
how well it can distinguish between different facial features. An XML file with the name haar cascade frontal 
face default.xml contains the training data that was utilised for this project. 


This module will retrieve individual photographs from the images folder one at a time. The ability to detect 
eye blinks is the first component of this module. It does this by employing a pre-trained model-shape 
predictor of 68 face landmarks from the dlib package. This predictor determines whether the 68 facial 
landmarks are present in each image. In order to determine the EAR (Eye Aspect Ratio) values of each image 
using the method, the facial landmarks that are located around the eye are taken into consideration. If the eye 
is kept open throughout the measurement, the EAR ratio will not change. When the eye closes, there is a 
significant drop in the EAR ratio. It is called a blink if there is a sustained deviation from this ratio across 
multiple frames. These findings are put to use to investigate the levels of fear and distraction experienced by 
the liar during the interrogation. 


According to the findings of the study, the typical number of times a human adult blinks during a conversation 
is 26 times per minute. An increased blink rate was seen in participants of an experiment who were feeling 
emotionally excited, anxious, or frustrated. 


A UML diagram is a diagram that is based on the Unified Modeling Language (UML). The purpose of a 
UML diagram is to visually represent a system along with its primary actors, roles, actions, artefacts, or 
classes in order to better understand, change, maintain, or document information about the system. 
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The photographs in the images folder are retrieved one at a time by this module. The detection of eye blinks 
the initial stage of this module. It does this by employing a pre-trained model-shape predictor of 68 face 
landmarks from the dlib package. This predictor determines which 68 facial landmarks are present in each 
image. When determining the EAR (Eye Aspect Ratio) values of each image, the facial landmarks located 
around the eye are taken into consideration using the algorithm. In conditions where the eye is kept open, the 
EAR ratio does not change. When the eye closes, the EAR ratio experiences a dramatic decrease. In the event 
that this ratio continues to diverge for multiple frames in a row, we call this a blink. The anxiousness and 
distraction levels of the liar are analysed using these outcomes as they occur during the interrogation. 


According to the findings of the research, the average number of times a human adult blinks their eyes during 
a discussion is 26. An increase in the blink rate has been seen in an experiment with persons who were 
experiencing emotional excitement, worry, or displeasure. 


A UML diagram is a diagram that is based on the Unified Modeling Language (UML). Its aim is to visually 
represent a system together with its primary players, roles, actions, artefacts, or classes in order to better 
understand, change, maintain, or document information about the system. 


During the experiment, we showed the 5 deceitful individuals certain questions, and their blink rate was 
calculated. The system then provided the output in the form of the number of blinks that occurred for each 
question that was correctly answered. By doing so, the one conducting the interrogation will determine 
whether or not the deception is lying. An accuracy of 97.2 percent has been attained by the suggested system. 


Conclusion 


The system will use face detection and an eye blink calculation to infer whether a candidate is being dishonest 
during an interview. This information can be used by the interrogator to analyse the blink threshold value and 
pinpoint the lar during the course of an interrogation. Possible future improvements include thermal 
monitoring, which would entail recording footage of the suspect while they respond to interrogation 
questions. An in-depth analysis of the suspect's dishonest behaviour might then be performed using this 
footage in conjunction with facial detection and eye blink rate. 
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